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Abstract — We consider the problem of quickest event detection 
with sleep-wake scheduling in small extent wireless sensor net- 
works in which, at each time slot, each sensor node in the awake 
state observes a sample and communicates the information to the 
fusion centre. The sensor nodes in the sleep state do not sample 
or communicate any information to the fusion centre, thereby 
conserving energy. At each time slot, the fusion centre, after 
having received the samples from the sensor nodes in the awake 
state, makes a decision to stop (and thus declare that the event 
has occurred) or to continue observing. If it decides to continue, 
the fusion centre also makes the decision of choosing the number 
of sensor nodes to be in the awake state in the next time slot. We 
consider three alternative approaches to the problem of choosing 
the number of sensor nodes to be in the awake state in time slot 
fc + 1, based on the information available at time slot k, namely, 

1) optimal control of Alk+i, the number of sensor nodes to 
be in the awake state in time slot fc + 1, 

2) optimal control of Qk+i, the probability of a sensor node 
to be in the awake state in time slot fc + 1, and 

3) optimal probability q that a sensor node is in the awake 
state in any time slot. 

In each case, we formulate the problem as a sequential decision 
process. We show that a sufficient statistic for detecting the event 
and choosing an optimal control at time k is the a posteriori 
probability of change Ilfc. Also, we show that the optimal stopping 
rule is a threshold rule on the a posteriori probability of change. 
We provide a partial characterisation of the optimal policies 
for choosing Mk+i or qk+i, and then explore these policies 
numerically. The optimal policy for Mk+i can keep very few 
sensors awake during the prechange phase and then quickly 
increase the number of sensors in the awake state when a change 
is "suspected." Among the three sleep-wake algorithms described, 
we observe that the total cost is minimum for the optimum control 
of Mk+i and is maximum for the optimum control on g. 

Index Terms — Bayesian change detection, sequential change 
detection with observation cost, sleep-wake scheduling 



I. Introduction 

Event detection (e.g., physical intrusion of a human into a 
secure region) is an important application of wireless sensor 
networks (WSNs). Events for which such a WSN is deployed 
are typically rare events, and hence, much of the energy of 



the sensor nodes gets drained away in the pre-event period. 
As sensor nodes are energy-limited devices, this reduces the 
utility of the sensor network. Thus, in addition to the problem 
of quickest event detection, we are also faced with the problem 
of increasing the lifetime of sensor nodes which we address 
in this paper by means of optimal sleep-wake scheduling of 
sensor nodes. 

A sensor node can be in one of two states, the sleep state or 
the awake state. A node in the sleep state conserves energy by 
switching to a low-power state. In the awake state, a sensor 
node can make measurements, perform some computations, 
and then communicate information to the fusion centre. For 
enhancing the utility and the lifetime of the network, it is 
essential to have optimal sleep-wake scheduling for the sensor 
nodes, while achieving the measurement and the inference 
objective of the WSN. 

We are interested in the quickest detection of an event with 
a minimal number of sensors in the awake state. A common 
approach to this problem is by having a fixed deterministic 
duty cycle for the sleep-wake activity. However, the duty cycle 
approach does not make use of the prior information about the 
event, nor the observations made by the sensors, and hence is 
not optimal. 

Hence, in this paper, we formulate the problem as one 
of optimum sequential change detection. In the classical 
change detection problem [l ], the decision maker after having 
observed each sample, has to make a decision to stop, or 
to continue observing the next sample. In such a situation, 
the decision maker is concerned only about minimising the 
detection delay while keeping the probability of false alarm 
bounded from above by a, a parameter of interest. However, 
in the kind of WSN application described above, there is an 
additional cost associated with generating an observation and 
communicating it to the decision maker, which we incorporate 
in our formulation. To the best of our knowledge, our work 
is the first to look at the problem of joint design of optimal 
change detection and sleep-wake scheduling. 
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A. Summary of Contributions 

We summarise the main contributions of this paper below. 
1) We provide a model for the sleep-wake scheduling of 
sensors by taking into account the cost per observation 
(which is the sensing + computation + communication 
cost) per sensor in the awake state and formulate the 
joint sleep-wake scheduling and quickest event detection 
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problem subject to a false alarm constraint, in the 
Bayesian framework, as an optimal control problem. We 
show that the problem can be modelled as a partially 
observable Markov decision process (POMDP). 

2) We obtain an average delay optimum stopping rule for 
event detection and show that the stopping rule is a 
threshold rule on the a posteriori probability of change. 

3) Also, at each time slot k, we obtain the optimal strategy 
for choosing the optimum number of sensors to be 
in the awake state in time slot k + 1 based on the 
sensor observations until time k, for each of the control 
strategies described as follows: 

(i) control of Mk+i, the number of sensors to be in 
the awake state in time slot k + 1, 

(ii) control of Qk+i, the probability of a sensor to be 
in the awake state in slot fc + 1, and 

(iii) constant probability g of a sensor in the awake state 
in any time slot. 

B. Discussion of the Related Literature 

In this section, we discuss the most relevant literature 
on energy-efficient detection. Censoring was proposed by 
Rago et al. in Q as a means to achieve energy-efficiency. 
Binary hypothesis testing with energy constraints was formu- 
lated by Appadwedula et al. in |3|. These schemes find the 
"information content" in any observation, and uninformative 
observations are not sent to the fusion centre. Thus, censoring 
saves only the communication cost of an observation. In our 
work, by making a sensor go to the sleep state, we save the 
sensing + computation + communication cost of making an 
observation. 

In related work (0], Wu et al. proposed a low duty cycle 
strategy for sleep-wake scheduling for sensor networks em- 
ployed for data monitoring (data collection) applications. In 
the case of sequential event detection, duty cycle strategies are 
not optimal, and it would be beneficial to adaptively turn the 
sensor nodes to the sleep or awake state based on the prior 
information, and the observations made during the decision 
process, which is the focus of this paper 

In lO, Zacharias and Sundaresan studied the problem of 
event detection in a WSN with physical layer fusion and power 
control at the sensors for energy-efficiency. Their Markov de- 
cision process (MDP) framework is similar to ours. However, 
in Q, all the sensor nodes are in the awake state at all time. 
In our work, we seek an optimal state dependent policy for 
determining how many sensors to be kept in the awake state, 
while achieving the inference objectives (detection delay and 
false alarm). 

C. Outline of the paper 

The rest of the paper is organised as follows. In Section |II] 
we formulate the sleep-wake scheduling problem for quickest 
event detection. We describe various costs associated with 
the event detection problem. Also, we outline various con- 
trol strategies for sleep-wake scheduling of sensor nodes. In 
Section HUl we discuss the optimal sleep-wake scheduling 
problem that minimises the detection delay when there is a 



feedback from the decision maker (in this case, the fusion 
centre) to the sensors. In particular, the feedback could be the 
number of sensors to be in the awake state or the probability 
of a sensor to be in the awake state in the next time slot. We 
show that the a posteriori probability of change is sufficient 
for stopping and for controlling the number of sensors to be 
in the awake state. In Section HVl we discuss an optimal open 
loop sleep-wake scheduler that minimises the detection delay 
where there is no feedback from the fusion centre and the 
sensor nodes. We obtain the optimal probability with which a 
sensor node is in the awake state at any time slot. In SectionlVl 
we provide numerical results for the sleep-wake scheduling 
algorithms we obtain. Section |VT] summarises the results in 
this paper 

II. Problem Formulation 

In this section, we describe the problem of quickest event 
detection with a cost for taking observations and set up the 
model. We consider a WSN comprising n unimodal sensors 
(i.e., all the sensors have the same sensing modality, e.g., 
acoustic, vibration, passive infrared (PIR), or magnetic) de- 
ployed in a region A for an intrusion detection application. We 
consider a small extent network, i.e., the region A is covered 
by the sensing coverage of each of the sensors. An event (for 
example, a human "intruder" entering a secure space) happens 
at a random time. The problem is to detect the event as early 
as possible with an optimal sleep-wake scheduling of sensors 
subject to a false alarm constraint. 

We consider a discrete time system and the basic unit 
of time is one slot. The slots are indexed by non-negative 
integers. A time slot is assumed to be of unit length, and hence, 
slot k refers to the time interval [fc, fc + 1). We assume that 
the sensor network is time synchronised (see, [6 1 for achieving 
time synchrony). An event occurs at a random time T e Z+ 
and persists from there on for all k ^ T. The prior distribution 
of T (the time slot at which the event happens) is given by 

PiT-fcl- / ^' if fc-0 

^ \ iffc>0, 

where < p < 1 and < p < 1 represents the probability that 
the event happened even before the observations are made. We 
say that the state of nature, Qk is before the occurrence of 
the event (i.e., Qk = for k < T) and 1 after the occurrence 
of the event (i.e., 6fc = 1 for fc > T). 

At any time fc e Z^, the state of nature Qk can not be 
observed directly and can be observed only partially through 
the sensor observations. The observations are obtained sequen- 
tially starting from time slot fc = 1 onwards. Before the event 
takes place, i.e., for 1 < fc < T, sensor i observes xj^^ e M 
the distribution of which is given by Fo{-), and after the event 
takes place, i.e., for k > T, sensor i observes X^*' e M the 
distribution of which is given by Fi{-) ^ Fo{-) (because of 
the small extent network, at time T, the observations of all the 
sensors switch their distribution to the postchange distribution 
Fi(-)). The corresponding probability density functions (pdfs) 
are given by /o(-) and /i(-) ^ /o('l!l Conditioned on the 

'if the observations are quantised, one can work with probability mass 
functions instead of pdfs. 
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State of nature, i.e., given the change point T, the observations 
X/^ s are independent across sensor nodes and across time. 
The event and the observation models are essentially the same 
as in the classical change detection problem, Q and IjS]. 

The observations are transmitted to a fusion centre. It 
is assumed that the communication between the sensors 
and the fusion centre is error-free and completes before 
the next measurements are takeiJl At time k, let A4k — 
{'ik,i,ik,2,- ■ ■ ,jfc,Mfc} Q {1,2,--- ,n} be the set of sensor 
nodes that are in the awake state, and the fusion centre 
receives a vector of Mk observations Yfe = ^fc^*" •= 
[Xl^''-'\xl^'''\ ■ ■ ■ At time slot k, based on the 

observations so far Y[i.fc]0 the distribution of T, /o(-)^ ^nd 
/i( ), the fusion centre 

1) makes a decision on whether to raise an alarm or to 
continue sampling, and 

2) if it decides to continue sampling, it determines the 
number of sensors that must be in the awake state in 
time slot fc + 1. 

Let Dk G {0, 1} be the decision made by the fusion centre to 
"continue sampling" in time slot k + 1 (denoted by 0) or "stop 
and raise an alarm" (denoted by 1). If = 0, the fusion 
centre controls the set of sensors to be in the awake state in 
time slot k + 1, and if Df^ — 1, the fusion centre chooses 
■Mk+i — 0. Let Ak G Ahe the decision (or control or action) 
made by the fusion centre after having observed Yfe at time 
k. We note that Ak also includes the decision D^- Also, the 
action space A depends on the feedback strategy between the 
fusion centre and the sensor nodes which we discuss in detail 
in Section ini] Let Ife := [Y[i.fc] , A[o.fc_i]] be the information 
available to the decision maker at the beginning of slot fc. 
The action or control A^ chosen at time k depends on the 
information 1^ (i.e., Ak is Ifc measurable). 

The costs involved are i) A^, the cost due to 
(sampling + computation + communication) per observation 
per sensor, ii) Xf, the cost of false alarm, and iii) the detection 
delay, defined as the delay between the occurrence of the event 
and the detection, i.e., (t — r)+, where r is the time instant at 
which the decision maker stops sampling and raises an alamfl. 
Let Ck : {0, 1} x {(0, 0), (0, 1), • • ■ , (0, n), (1, 0)} M+ be 
the cost incurred at time slot fc. For fc ^ r, the one step 
cost function is defined (when the state of nature is 0^, the 
decision made is Dk, and the number of sensors in the awake 
state in the next time slot is Mk+i) as 

Ck{ek,Dk,Mk+i) 



(1) 



^This could be achieved by synchronous time division multiple access, with 
robust modulation and coding. For a formulation that incorporates a random 
access network (but not sleep-wake scheduling), see |9| and |10|. 

^The notation yjfcj-fc,] defined for ki < k-z means the vector 

^We note here that the event {r = fc} is completely determined by the 
information 1^;, and hence, r is a stopping time with respect to the sequence 
of random variables Ii, I2, ■ ■ ■ . 
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= l,Dk 
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and for fc > r, Cfe (•, -, •):= 0. Note that in the above definition 
of the cost function, if the decision Dk is 1, then Mk+i is 
always 0. For time k ^t, the cost Ck{Qk, Dk, Mk+i) can be 
written as 

Ck{Qk,Dk,Mk+i)= \f ■ l{efc=o}l{Dfc=i} 

+ (l{efc=i} + AsMfc+i) l{_Dfc=o}-(2) 

We are interested in obtaining a quickest detection proce- 
dure that minimises the mean detection delay and the cost of 
observations by sensor nodes in the awake state subject to the 
constraint that the probability of false alarm is bounded by a, 
a desired quantity. We thus have a constrained optimization 
problem. 



mmimise 



(r - T)+ + A, ^ Mk 
subject to P {t < T} < a 



k=l 



(3) 



where r is a stopping time with respect to the sequence 
Ii , I2 , • • • . The above problem would also arise if we imposed 
a total energy constraint on the sensors until the stopping time 
(in which case, A^ can be thought of as the Lagrange multiplier 
that relaxes the energy constraint). Let A/ be the cost of false 
alarm. The expected total cost (or the Bayes risk) when the 
stopping time is r is given by 



R{t) = A/P{T<r} + E ir-T)+ + \sY,Mk 

k=l 

T-1 

A/l{e,=o} + X! (^{©lo^i} + AsAffc+i) 

fc=0 

T-l 

c,(e,,i,o) + ^cfc(efc,o,A4+i) 

fc=0 

T 

^Ck{ek,Dk,Mk+i) 

00 

"^Ck{ek,Dk,Mk+i) 

.k=0 

00 

Y,^[ck{&k,Dk,Mk+i)] 



(4) 



= E 



(a) 



(5) 



fc=0 



where step (a) follows from Ck{- ,■,■)— for k > t, and step 
(&) follows from the monotone convergence theorem. Note 
that A/ is a Lagrange multiplier and is chosen such that the 
false alarm constraint is satisfied with equality, i.e., Pfa = ct 
(see Q). 

We note that the stopping time t is related to the control 
sequence {Ak} in the following manner For any stopping 
time T, there exists a sequence of functions (also called a 
policy) ly = {vi, 1^2, ■ ■ ■) such that for any fc, when t — k, 
Dk' = n-(Ifc') = for all fc' < fc and Dk' = i^k'ih') = 1 
for all fc' ^ fc. Thus, the unconstrained expected cost given by 
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Eqn. |4]is 

i?(r) 



(a) 



k=0 

E 



(6) 



^E[cfc(efc,Z?fe,Mfe+i)] 

fc=0 

oo 

^ E[cfe(6fc,i^fc(Ifc), Affc+i)] 

fc=0 
oo 

j2mck{Qk,i^k{ik),Mk+i) \ ik]] 

3 

CO 

^E[cfe(efe,i^fe(Ife),Mfe+i) |Ifc] 

.fc=0 
r 

^E[cfe(e,,i/fe(Ife),Affe+i) |Ifc] 
.fc=o 

where step (a) above follows from the monotone convergence 
theorem. From Eqn. |2] it is clear that for fc ^ r 

E[cfc(efe,i/fe(Ife),A4+i) I Ifc] 

+ ^[(l{e»= = l} + '^^sA^+l) • l{;.fc(lfc)=0} I Ife] 

= ^/ ■ E[l{e,=o} I Ife] ■ 

+ (E[l{e.=i} I Ife] + A. ■ E[Mfc+i I I,]) • l{.,(i,)=o} 

For k ^ T, define the a posteriori probability of the 
change having occurred at or before time slot fc, life := 
E[l{efc=i} I Ife]' hence, we have 

E[cfc(efc,iyfe(Ifc),Affe+i) I Ife] 

= A/ • (1 - nfc)i{^j^(i^)^i} 

+ (Hfc + A, • E[Mfe+i I Ife.]) l{,,(i,)=o}- (7) 
Thus, we can write the Bayesian risk given in Eqn. |6]as 

r-l 

R{t) = E Xj.{l-Ur)+Y, (Hfe + A,E[Affe.+i I Ife]) (8) 

fe=o 

We are interested in obtaining an optimal stopping time t and 
an optimal control of the number of sensors in the awake state. 
Thus, we have the following problem. 



minimise E 



r-l 



A/ • (1 - n,) + ^ (Hfc + A,E[Affc+i I Ife]) 



fe=0 



(9) 



We consider the following possibilities for the problem defined 
in Eqn. |9] 

1) Closed loop control on Affe+i: At time slot k, the 
fusion centre makes a decision on Affe+i, the number 
of sensors in the awake state in time slot k + 1, based 
on the information available (at the fusion centre) up to 
time slot k. The decision is then fed back to the sensors 
via a feedback channel. Thus, the problem becomes 

T-l 

A^(i-n,) + ^(nfe + A,A/fe.+i 



min E 



fe=0 



the information Ife. qk+i is then broadcast via a feedback 
channel to the sensors. Thus, given Ife, the number of 
sensors in the awake state Mk+i, at time slot fc + 1, 
is Bernoulli distributed with parameters {n,qk+i) and 
E[A/fe-|_i I Ife] — nqk+i- Thus, the problem defined in 
Eqn. |9] becomes 



mm 



A/(l - n^) + XI (^fe + A,,ngfe+i) 



fe=0 



(11) 



3) 



Open loop control on q: Here, there is no feedback 
between fusion centre and the sensor nodes. At time 
slot fc, each sensor node is in the awake state with 
probability q. Note that A/fe, the number of sensors in the 
awake state at time slot k is Bernoulli distributed with 
parameters {n,q). Also note that {Affe} process is i.i.d. 
and E[A/fe+i | Ife] = nq (also, Mk+i is independent of 
the information vector Ife). Note that the probability q is 
constant over time. Thus, the problem defined in Eqn. |9] 
becomes 



min E 



A/(l 



r-l 

no + X (Hfe 

fc=0 



Xgnq) 



(12) 



Here, q is chosen (at time fc = 0) such that it minimises 
the above cost. 

Note that the first two scenarios require a feedback channel 
between the fusion centre and the sensors whereas the last 
scenario does not require a feedback channel. 

In Section |III1 we formulate the optimization problem 
defined in Eqns. [TO] and [TT] in the framework of MDP and 
study the optimal closed loop sleep-wake scheduling policies. 
In Section HV] we formulate the optimization problem defined 
in Eqn. [12] in the MDP framework and obtain the optimal 
probability q of a sensor in the awake state. 

III. Quickest Change Detection with Feedback 

In this section, we study the sleep-wake scheduling problem 
when there is feedback from the fusion centre to the sensors. 

At time slot fc, the fusion centre receives a A/fe -vector 
of observations Yfe, and computes life. Recall that life = 



P<^T < fc 



(10) 



2) Closed loop control on gfe+i: At time slot fc, the fusion 
centre makes a decision on qk+i, the probability that a 
sensor is in the awake state at time slot fc + 1 based on 



Ife I is the a posteriori probability of the event hav- 
ing occurred at or before time slot fc. For the event detection 
problem, a sufficient statistic for the sensor observations at 
time slot fc is given by life (see lITTl and page 244, 1 12|). When 
an alarm is raised, the system enters into a terminal state 't'. 
Thus, the state space of the {life} process is 5 = [0, 1] U {t}. 
Note that life is also called the information state of the system. 

In the rest of the section, we explain the MDP formulation 
that yields the closed loop sleep-wake scheduling algorithms. 

A. Control on the number of sensors in the awake state 

In this subsection, we are interested in obtaining an optimal 
control on Affe+i, the number of sensors in the awake state, 
based on the information we have at time slot fc. 

At time slot fc, after having observed X^*", the fusion 
centre computes the sufficient statistic life. Based on life, 
the fusion centre makes a decision to stop or to continue 
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sampling. If the decision is to continue at time slot k + 1, 
the fusion centre (which also acts as a controller) chooses 
Mfc+i, the number of sensors to be in the awake state at time 
slot k + 1. The fusion centre also keeps track of the residual 
energy in the sensor nodes, based on which it chooses the 
set of sensor nodes J^k+i that must be in the awake state 
in time slot fc + 1. Since, the prechange and the postchange 
pdfs of the observations are the same for all the sensor nodes 
and at any time, the sensor observations are conditionally 
independent across sensors, any observation vector of size 
TO has the same pdf and hence, for decision making, it is 
sufficient to look at only the number of sensors in the awake 
state Mk+i, i.e., the indices of the sensor nodes that are in 
the awake state are not required for detection (we assume that 
the fusion centre chooses the sequence A^i, A^2, ■ • • in such 
a way that the rate at which the sensor nodes drain their 
energy is the same). Thus, the set of controls at time slot 

k is given by ^ = |(stop, 0), U„e{o,i,... ,„}(continue, m) 

= {(1,0),(0,0),(0,1),--- ,(0,n)}. 

We show that 11^ can be computed in a recursive manner 
from the previous state Hk-i, the previous action Ak-i, and 
the current observation X^'' as. 



policy fi = ■■■) as 



i?(r) 



^c{Uk,fik(lk)) 

k=0 
oo 

Y,c{nk,M^k)) 



.k=0 



(15) 



Since life is a sufficient statistic for Ife, for any policy /i^ there 
exists a corresponding policy Jik such that JlkiHk) ~ Mfe(Ife)' 
and hence, the last step in the above equation follows (see page 
244, lfT2l ) Since, the one stage cost and the density function 
02 (y; Ilfe-i) are time invariant, it is sufficient to consider the 
class of stationary policies (see Proposition 2.2.2 of |13|). Let 
11 : S ^ Ahe a stationary policy. Hence, the cost of using 
the policy Ji is given by 



^c(nfe,/i(nfe)) 



Lfc=0 



Ho = TTo 



and hence, the minimal cost among the class of stationary 
policies is given by 



J*(7ro) = min E 



*(nfe_ 
t, 



J2ci'nk,mk)) 



.k=0 



,A 



fe-i, 



) 

if nfe_i = t 

if Ak-i ^ 1 

(13) 

if nfe._i e [o,i],Ak-i = (o,Mfe) 



The dynamic program (DP) that solves the above problem is 
given by the Bellman's equation. 



where 



Ilk 



^0 (X^) 

h (xf^n 



:= nfe + (i-nfe)p, 

:= n /o(4'^ 



ieMk 

n0i (x^^) + (i-n)</>o (xf'')(i4) 



Thus, the a posteriori probability process {life} is a controlled 
Markov process. Note that Efe = life + (1 - Ilk)p = Epfe+i] 
is observed. Motivated by the structure of the 



J*{tt) = min|c(7r,l),Fj.(7r)| (16) 
where the function iJj. : [0, 1] -> R+ is defined as 

min{?(^, (0, to)) + E^.f^^^) [J* (<l>(7r, (0, to), Y))] }(17) 

0<m<n 

where Y and y are TO-vectors. The notation E02(y;#) [•] means 
that the expectation is taken with respect to the pdf 4>2{y", tt) 
(recall Eqn. [14] for the definition of 02 (y; tt)). Thus, Eqn. [16] 
can be written as 

J*(7r) = min{A/ • (1 -7r),7r + A7-(7r)} (18) 

where the function Aj* : [0, 1] — > K+ is defined as 



Mk 
k+1 



Ajt{n) — min \ Xf^m 

Q<ni<n 



before X 

cost given in Eqn. [T] we define the one stage cost function 
c : S X IR.+ when the (state, action) pair is (s,a) as 



-02(y;7r) 



J* 



(19) 



c(s,a) = 




if s = 7re [0, l],a= (1,0) 
if s = TT e [0, 1], a = (0, to) 
if s = t. 



Since Mk+i is chosen based on the information Ife, there exists 
a function I'l such that Mk+i = t^fe(Ife). Thus, the action 
or control at time k is given by ^fc(Ifc) = (i^fc(Ife), (/^(Ife)). 
Hence, we can write the Bayesian risk given in Eqn. [4] for a 



The optimal policy /i* that achieves J* gives the optimal 
stopping rule, t*, and the optimal number of sensors in the 
awake state, Mj*, M2*, • • ■ , M;,. 

We now establish some properties of the minimum total cost 
function J*. 

Theorem 1: The total cost function J*(tt) is concave in tt. 

Also, we establish some properties of the optimal policy 11* 
(which maps the a posteriori probability of change life to the 
action space A) in the next theorem. 
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Theorem 2: The optimal stopping rule is given by the 
following threshold rule where the threshold is on the a 
posteriori probability of change. 



T* = inf{fc : Uk ^ r}, 



(20) 



for some F e [0, 1]. The threshold F depends on the probabil- 
ity of false alarm constraint, a (among other parameters like 
the distribution of T, /o, /i). 

Theorem |2] addresses only the stopping time part of the 
optimal policy jj* . We now explore the structure of the optimal 
closed loop control policy for M* : [0, 1] — ?► Z+, the optimal 
number of sensors in the awake state in the next time slot. 
At time k, based on the (sufficient) statistic Flfc, the fusion 
centre chooses M^^j^ = M*{Iik) number of sensor nodes in 
the awake state. For each < m < n, we define the functions 



[0,1] 



and A 



(rn) 
J* 



[0,11 



-02(y;ir) 



and A^j^'{n) := A.m + Syf^W 



J* 



(m) / 



l(Y) 



(tt) and A ™ (tt) are concave 

,(m) 



We have shown in the proof of Theorem [T] that for any m 
0, 1, 2, • • • , n, the functions ---i 'i'™)' 

in TT. 

Theorem 3: For any tt G [0,1], the functions Bjl^'{n) 
monotonically decrease with m. 

Remark: By increasing the number of sensor nodes in the 
awake state, i.e., by increasing m, we expect that the a 
posteriori probability of change will get closer to 1 or closer to 
(depending on whether the change has occurred or not). In 
either case, the one stage cost decreases, and hence, we expect 
that the functions B^™\Tr) monotonically decrease with m. 

At time k, (life) can be thought of as the cost-to-go 
function from slot fc + 1 onwards (having used m sensor nodes 
at time fc+1). Note that A^J^"^ (tt) has two components, the first 
component A^m increases with m and (from Theorem |3]l the 
second component decreases with m. As m takes values in a 
finite set {0, 1, 2, • • • , n^, for each tt, there exists an optimal 
M*{tt) for which A^ff '"'^^\tt) is minimum. For any given 
TT G [0, 1], we define the differential cost d : {1, 2, • • • , n} ^ 
M+ as 



d{m; tt) = B 



(m-l) 



(vr)-i?(?)(vr) 



(21) 



Note that for any 1 < m < n, d{m; tt) is bounded and 
continuous in vr (as B^j^\ are bounded and concave in vr). 
Also note that d(m; 1) = as = B^?^(l) = 0. 

We are interested in d{m;TT) for tt e [0, F). In Figure [T] we 
plot d{m\TT) against tt for rn = 1,2, and 3 (for the set of 
parameters n — \f — 100, — 0.5, and /o and /i are 
unit variance Gaussian pdfs with means and 1 respectively). 
We observe that d{m; tt) monotonically decreases in m, for 
each TT G [0,r) (i.e., d(l;7r) > d(2;7r) > d(3; tt)). We have 
observed this monotonicity property for different sets of ex- 
periments for the case when /o and /i belong to the Gaussian 
class of distributions. We conjecture that this monotonicity 
property of d holds and state the following theorem which 
gives a structure for M*, the optimal number of sensors in 
the awake state. 



o 
o 



o 

0) 
3= 
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0.2 0.4 0.6 
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Fig. 1. Differential costs, (i( ;7r), for n - 
0.5, /o ~ A^(0, 1) and /i ~ A^(l, 1). 



10 sensors, Aj = 100.0, As 



Theorem 4: If for each vr e [0, F), d(m; tt) decreases 
monotonically in m, then the optimal number of sensors in 
the awake state, M* : [0, 1] — > {0, 1, • • • , n\ is given by 

M*(tt) = maxjm : d{m\TT) > A^}. 

B. Control on the probability of a sensor in the awake state 

In this subsection, we are interested in obtaining an optimal 
control on qu+i, the probability of a sensor in the awake state, 
based on the information we have at time slot fc, instead of 
determining the number of sensors that must be in the awake 
state in the next slot. 

At time slot fc, after having observed X^*^, the fusion centre 
computes the sufficient statistic 11^, based on which it makes a 
decision to stop or to continue sampling. If the decision is to 
continue at time slot fc+1, the fusion centre (also acts as a con- 
troller) chooses <7fc+i, the probability of a sensor to be in the 
awake state at time slot fc + 1. Thus, the set of controls at time 



slot fc is given by ^ = < (stop, 0), Ugg[o.i] (continue, g) > = 



l,U,e[o,i](0,g)| ={(l,0),{0}x [0,1]}. 

When the control Ak = {0,qk+i) is chosen, A/fe+i, the 
number of sensors in the awake state at time slot fc + 1 is 
Bernoulli distributed with parameters (n, <7fc+i). Let ■y,n{qk+i) 
be the probability that m sensors are in the awake state at time 
slot fc + 1. 7r„((7fc+i) is given by 



(22) 



The information state at time slot fc 11^, can be computed in a 
recursive manner from Ilfc-i, Ak-i and X^*" using Eqn. [T3] 
Thus, it is clear that the {Uk} process is a controlled Markov 
process, the state space of the process being S = [0, 1] U {t}. 
Motivated by the cost function given in Eqn. |7] define the one 
stage cost function c (s, a) when the (state, action) pair is (s, a) 
as 

"A/(l-7r), if s TT e [0, l],a = (1,0) 
c(s, a) — ^ TT + Xsuq, if s = TT g [0, 1], a = (0, q) 
0, if s = t. 
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Since, the one stage cost and the density function 02 (y; ^k-i) 
are time invariant, it is sufficient to consider the class of sta- 
tionary poHcies (see Proposition 2.2.2 of [13]). Let Jl : S A 
be a stationary poUcy. Hence, the cost of using the policy 
is given by 



.k=0 



Ho = ttq 



and hence the minimal cost among the class of stationary 
policies is given by 



riTTo) 



rnin E 



^c(nfe,/i(n,)) 



.fe=0 



Tin 



The DP that solves the above problem is given by the 
Bellman's equation. 



J* in) 

where Hj. : [0, 1] - 

Hr* in) 

mm < c(iT. (0, q)) 
0<q<i ' ■ ^ ' 



min{c(7r, l),Hj. (tt)} 
is defined as 

n 

- J2 7m(g)E^,(y.^)[J-(<I.(7r,(0,m),Y)] 



We choose q that minimises the Bayesian cost given by 
Eqn.in] 

At time slot fc, the fusion centre receives a vector of 
observation X^*" and computes Ilfc. In the open loop scenario, 
the state space is 5 = { [0, l]U{t}}. The set of actions is given 
by ^ = {stop, continue} = {1,0} where '1' represents stop 
and '0' represents continue. Note that life can be computed 
from Ilk-i, Ak-i, and in the same way as shown in 

Eqn.[T3] Thus, {life}, k G Z_|_ is a controlled Markov process. 
Motivated by the structure of the cost given in Eqn. |7] we 
define the one stage cost function c : S x A ^ R+ when the 
(state, action) pair is (s,a) as 

A/(l-7r), if s = TT e [0,l],a = 1 
c(s, a) — ^ n + Xsnq, if s = tt £ [0, 1], a = 
0, if s = t. 

Since, the one stage cost and the density function 02 (y; Hfe-i) 
are time invariant, it is sufficient to consider the class of sta- 
tionary policies (see Proposition 2.2.2 of [13]). Let Jl : S A 
be a stationary policy. Hence, the cost of using the policy Ji 
is given by 



where Y and y are m-vectors. Recall that the expectation is 
taken with respect to the pdf 02(y; t^)- The Bellman's equation 
can be written as 

J*(7r) = min{A/ • (1 -7r),7r + Aj.(7r)} (23) 

where the function Aj* : [0, 1] — > K+ is defined as 

Aj.{tt) 

'^■0i(Y 



.k=0 



?(nfc,/i(nfe)) 



Ho = TTo 



and the optimal cost under the class of stationary policies is 
given by 



J*(vro) 



rnin E 



.fc=o 



min < XsTiq - 



E 

m=0 



7m(g)E02(y;#) 



J* 



The DP that solves the above equation is given by the 
Bellman's equation. 



The optimal policy /i* gives the optimal stopping time r*, and 
the optimal probabilities, g^, k = 1, 2, • • • , r*. The structure 
of the optimal policy is shown in the following theorems. 

Theorem 5: The total cost function J*{tt) is concave in vr. 

Theorem 6: The optimal stopping rule is a threshold rule 
where the threshold is on the a posteriori probability of change, 

T* - inf{fc : Hfe ^ r}, 

for some F e [0, 1]. The threshold F depends on the probabil- 
ity of false alarm constraint, a (among other parameters like 
the distribution of T, /o, /i). 

IV. Quickest Change Detection without Feedback 

In this section, we study the sleep-wake scheduling problem 
defined in Eqn. [12] Open loop control is applicable to the 
systems in which there is no feedback channel from the fusion 
centre (controller) to the sensors. Here, at any time slot fc, a 
sensor chooses to be in the awake state with probability q 
independent of other sensors. Hence, {A/^}, the number of 
sensors in the awake state at time slot k is i.i.d. Bernoulli 
distributed with parameters {n,q). Let 7™ be the probability 
that m sensors are in the awake state. 7™ is given by 



J*{n) = min|c(7r, l),iJj.(7r] 
where : [0, 1] — > R+ is defined as 



J*U(7r,(0,m),Y) 



where Y and y are m-vectors. The above equation can be 
written as 



J*{tt) = min{A/ • (1 -7r),7r + Aj. (tt)} 
where the function Aj* : [0, 1] — > R+ is defined as 



(25) 



Aj. (tt) ^Xsnq+ > 7mE02(y;*) 



m=0 



J* 



q'^il-qy 



(24) 



The optimal policy fi* that achieves J* gives the optimal 
stopping rule, r*. We now prove some properties of the 
optimal policy. 

Theorem 7: The optimal total cost function J*(7r) is con- 
cave in TT. 

Theorem 8: The optimal stopping rule is a threshold rule 
where the threshold is on the a posteriori probability of change, 

T* = inf{fc : Hfe ^ F}, 
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Posterior probability, it Time slot, k 



Fig. 2. Optimum number of sensors in tlie awake state M* for n = 10 
sensors, A/ = 100.0, = 0.5, /o - A^(0, 1) and /i ~ J\f{l, 1). Note 
tliat r = 0.9 corresponds to the thresliold. 



Fig. 3. A sample run of event detection witli ra = 10 sensors, Aj = 100.0, 
A^ = 0.5, /o ~ A/'(0, 1) and /i - 1). 



for some F G [0, 1]. The threshold F depends on the probabil- 
ity of false alarm constraint, a (among other parameters like 
the distribution of T, /o, /i). 

For each q G [0, 1], we compute the optimal mean detection 
delay Eqd (as a function of q), and then find the optimal q* 
for which the optimal mean detection delay is minimum. 

V. Numerical Results 

We compute the optimal policy for each of the sleep-wake 
scheduling strategies given in Eqns. [18] |23] |25] using value- 
iteration technique (see lfT2l ). We consider n — \Q sensors. 
The distributions of change-time T is taken to be geometric 
(0.01) (and ttq = 0). Also, the prechange and the postchange 
distributions of the sensor observations are taken to be A/'(0, 1) 
and A/'(l, 1). We set the cost per observation per sensor, to 
0.5 and the cost of false alarm. A/ to 100.0 (this corresponds 
to a = 0.04). 

• Optimal control of A/fc+i: 

We compute M* the optimal number of sensors to be 
in the awake state in time slot fc + 1 as a function 
of the a posteriori probability of change tt (from the 
optimal policy fi* given by EqnlTSIl by the value iteration 
algorithm iTSl . lfT4l and plot in Figure |2] We note that 
in any time slot, it is not economical to use more than 3 
sensors (though we have 10 sensors). Also, from Figure |2] 
it is clear that M* increases monotonically for tt < 0.6 
and then decreases monotonically for tt > 0.6. Note that, 
the region tt £ [0.5, 0.82] requires many sensors for 
optimal detection whereas the region [0.0, 0.3]U[0.9, 1.0] 
requires the least number of sensors. This is due to the 
fact that uncertainty is more in the region tt G [0.5, 0.82] 
whereas it is less in the region [0.0, 0.3] U [0.9, 1.0]. 

In Figure [3] we plot the trajectory of a sample path of Hfc 
versus the time slot k. In our numerical experiment, the 
event occurs at T = 152. When the number of sensors 
to be in the awake state Mk+i is M*{TTk) (taken from 
Figure IIJ, for a threshold of 0.9, we see that the detection 
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Fig. 4. Total cost J{tt) for n = 10 sensors. A/ = 100.0, As = 0.5, 
fo ~ A^(0, 1) and /i ~ Af{l, 1). Note that the threshold corresponding to 
A/ = 1 is 0.895, for M = 2 is 0.870, for M = 3 is 0.825, and for M* is 
r = 0.9. 



happens at t^/. = 161. When Alk+i = 10 sensors 
(no sleep scheduling), we find the detection epoch to be 
Tio = 153. When M^+i = 3 sensors (we chose 3 because 
M* < 3), the stopping happens at T3 = 156. From the 
above stopping times, it is clear that the detection delay 
does not vary significantly in the above three cases. By 
having an optimal sleep-wake scheduling, we observe 
that until the event occurs only one sensor is in awake 
state and as soon as the event occurs, the sleep-wake 
scheduler ramps up the number of sensors to 3, thereby 
making a quick decision. Thus, the optimal sleep-wake 
scheduling uses a minimal number of sensors before 
change and quickly ramps up the number of sensors after 
change for quick detection. Also, we see from Figure |3] 
that the iTk trajectory corresponding to Mfe_|_i(7r) — 10 
(and Mfe+i(7r) = 3) gives more reliable information 
about the event than the iTk trajectory corresponding to 
Mfe+iW = M*. 

We also plot the total cost function J(7r) for the above 
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Fig. 5. Total cost J*(7r) for n = 10 sensors, = 
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Fig. 6. Optimum probability of a sensor in the awake state, (i'^^]^{7r) for 
100.0, As = 0.5, fo ~ Af{0, 1) and /i ~ A^(l, 1). 



10 sensors, A 



cases in Figure|4] Though the detection delays do not vary 
much, the total cost varies significantly. This is because 
the event happens at time slot T = 152. In the case 
of Mk+i — M*, it is clear from Figures |2] and |3]that 
only one sensor is used for the first 158 time slots. This 
reduces the cost by 10 times compared to the case of 
Mk+i — 10 (in this sample path) and about 3 times 
compared to the case of Mk+i — 3 (in this sample path). 
We note from Figure H] that it is better to keep 3 sensors 
active all the time than keeping 10 sensors active all the 
time. Also, in the case of Mk+i = 1, after the event 
occurs, the a posteriori probability takes more time to 
cross the threshold compared to the optimal sleep-wake 
(which quickly ramps up from 1 to 3 sensors) and hence, 
the total cost corresponding to M^+i = 1 is slightly 
worse than that of M^+i = M*. 

Optimal control of qk+i'- In the case of control on q^, 
we consider the same set of parameters as in the case of 
control on Mk- We computed the optimal policy from the 
DP defined in Eqn. |23] by value iteration. The optimal 
policy also gives the optimal probability of choosing a 
sensor in the awake state, ql^i- We plot the total cost 
J* (tt) in Figure |5] We also plot the optimum probability 
of a sensor in the awake state, q*{TT) in Figure |6] We 
observe that for tt < 0.72, g*(7r) is an increasing function 
of TT, and for tt > 0.72, g*(7r) decreases with tt. This 
agrees well with the intuition for the optimal control on 
Mfe+i. 

Open loop control on q: 

We consider the same set of parameters for the case of 
open loop control on q. We obtain J*(0) for various 
values of q and plotted in the Figure [T] We obtain the 
plot for As = 0.5 and for — 0.0. In the special case 
of q = 1, i.e., having Mk+i = 10 sensors, and with 
Xs = 0.5, we observe that the total cost is 100 which 
matches with the corresponding cost in Figure 2] Also, 
in the limiting case of g — > 0, all the sensor nodes are in 
the sleep state at all time slots, and the detection happens 
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Fig. 7. Total cost J*(0) for n = 10 sensors. Ay 
and /i ~ A^(l,l). 



100.0, fo ~ M(0, 1) 



only based on Bayesian update (i.e., based on the prior 
distribution of T). Thus at q 0, the total cost is the 
same (which is 73) for As = 0.5 and As — 0.0 which is 
also evident from Figure |7] 

Note that when As > 0, for low values of q, the detection 
delay cost dominates over the observation costs in J* (0) 
and for high values of q, the observation costs dominate 
over the detection delay cost. Thus, there is a trade-off 
between the detection delay cost and the observation costs 
as q varies. This is captured in the Figure |7] Note that 
the Bayesian cost is optimal at q = 0.15. When As = 0, 
as q increases the detection delay decreases. Hence, we 
see the monotonically decreasing trend for As = 0.0. 

From Figures |4] |5] and |7] we note that the total cost J{tt) 
is the least for optimal control on Mk+i- Also, we note that in 
the open loop control case, the least total cost J*(0) = 55 is 
achieved when the attempt probability, q is 0.15 (see Figure |2l 
this corresponds to an average of 1.5 sensors being active). 
It is to be noted that this cost is larger than that achieved by 
the optimal closed loop policies (J*(0) — 50 for the closed 
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loop control on qk+i and J*(0) = 38 for the closed loop 
control on Mk+i). From Figures |3] and |2] we see that when 
Affc+i(7r) = M*{tt), the switching of the sensors between 
sleep and awake states happen only in 2 slots out of 161 slots. 
Otherwise only 1 sensor is on. 

VI. Conclusion 

In this paper, we formulated the problem of jointly opti- 
mal sleep-wake scheduling and event detection in a sensor 
network that minimises the detection delay and the usage of 
sensing/communication resources. We have set out to solve 
the problem in Eqn. |9] We have derived the optimal control 
for three approaches using the theory of MDP. We showed the 
existence of the optimal policy and obtained some structural 
results. 

We prescribe the sleep-wake scheduling poUcies as follows: 
When there is a feedback between the fusion centre and the 
sensors and if the feedback is unicast, it is optimal to use the 
control on Affc+i policy; when the feedback is only broadcast, 
then it is optimal to use the control on qk+i- If there is 
no feedback between the fusion centre and the sensors, we 
prescribe the open loop control on q policy. 

Appendix 

Proof of Theorem [T] 

We use the following Lemma to prove Theorem [T] 
Lemma 7.- If / : [0, 1] — >■ M is concave, then for any x S 
(for any m G Z+), the function /i : [0, 1] ^ M defined by 

#i(X) 



Ky) 



-02(x;i/) 



/ 



^#i(X) + (l-y)0o(X), 

is concave in y, where (/)i(x) and 0o(x) are pdfs on X, and 
02 (x; y) = #i(x) + (1 - y)0o(x). 

Proof For any given x, define the function hi : [0,1] — > K 
as 



/ii(y;x) 



/ 



#i(x) 



.y(t>im + (1 - 2/)'/'o(x) 

As T := J • ■ • dx is a linear operator and h{y) — Thi{y;x), 
it is sufficient to show that hi{y;x.) is concave in y. If f{y) 
is concave then (see |15|) 

f{y) = inf {aiy + b^} 

{ai,bi)el 

where / = {{a,b) eR"^ :ay + b> f{y),y 6 [0, 1]}. Hence, 
/ii(y;x) 

y'/'i(x) 



=/ 



2/'/'i(x) + (l-2/)(/>o(x) 

#i(x) 



inf < a, , , , N / s , , 
{a.M)ei [ \y<l>i{x) + (1 - y)<l>o{x 



2/01 (x) + (1 - 2/)'/'o(x) 



#i(x) + (1 - y)0o(x) 

inf \aiy(t)i{x) +bi y(t)i{x) + (1 - y)(t)Q{x.) \ 
inf \({ai + 6j)0i(x) - 6j0o(x) )y + 6j0o(x) \ 



which is an infimum of a collection of affine functions of y. 
This implies that hi{y,x) is concave in y (see fT5|). ■ 

The optimal total cost function J*{tt) can be computed 
using a value iteration algortithm. Here, we first consider a 
finite X-horizon problem and then we let fc — > oo, to obtain 
the infinite horizon problem. 

Note that the cost-to-go function, J^{tt) = A/ • (l — tt) is 
concave in tt. Hence, by Lemma[T] we see that the cost-to-go 
functions J^_i(7r), J^_2('^)' ' ' ' > "^a^ i'^) concave in tt. 
Hence for < A < 1, 



J*(7r)= lim Jo^W 
J*(A7ri + (l-A)7r2) = lim Jg^ f Atti + (1 - A)7r2) 



> lim A Jo« (TTi) + lim (1 - X)J^ (^2) 

K—¥oo A— ^00 

= AJ*(7ri) + (l-A)J*(^2) 
It follows that J*(7r) is concave in tt. ■ 

Appendix 

Proof of Theorem |2] 

Define the maps C : [0, 1] -> R+ and H : [0, 1] ^ M+, as 

C{tt) := Xf .{1-tt) 
H{tt) tt + Aj.{tt) 

Note that C(l) = 0, H{1) = 1, C(0) = A/ and i7(0) = 
Aj. (0). Note that 



Aj.(O) 



mm < Xf.in 

< < n 



2(X("); 



0<rn<n 



< min < Xsin + J* E 



-02(X('"):p) 



p-0i(XM) 
02 (XM;^) 
p-0i(XM) 



02 (XW;^) 



= mill {Asm + J*(p)} 

0<7n<n 

= -rip) 

< Xf -{l-p) (from Eqn. 16) 

The inequality in the second step is justified using Jensen's 
inequality and the inequality in the last step follows from the 
definition of J*. 

Note that H{1) - C(l) > and H{0) - C(0) < 0. As the 
function H{tt) — C{tt) is concave, by the intermediate value 
theorem, there exists T G [0, 1] such that H{T) = C{T). This 
r is unique as H{tt) = C{tt) for at most two values of tt. If in 
the interval [0,1], there are two distinct values of tt for which 
H{tt) = C(7r), then the signs of iJ(0)-C(0) and i7(l)-C(l) 
should be the same. Hence, the optimal stopping rule is given 
by 

T* = inf {k : Uk > r} 
where the threshold T is given by T + Aj, (T) = A/ • (l - F) . 
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Proof of Theorem |3] 

Define 



,(x('")) 
u 

V 



n/,(2:W), J =0,1. 

1=1 

(.T('"+i),a;(™+2),... 

7r'/'i(u) 

7r0i(u) + (1 - 7r)0o(u) 



Note that 



J* 



7^-0l(x(')) 



Ri V '/'2(x(');7r) 

71-01 (v) 



02 (x«;*) 



J* 



< 



J* 



02(v;7r) 
71-01 (v) 



/ J* (7r)02(u;7r)du 



'2(v;7r) dv02(u;7r) du 
02(v;7r) dv ) 02(u;-n-) du 



As J* is concave, the inequality in the second hne follows 
from Jensen's inequality. Hence proved. ■ 
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Appendix 

Proof of Theorem |4] 

Eqn. [19] and the monotone property of d{m; .) proves the 
theorem. ■ 



Appendix 

Proof of Theorem |5] 

Follows from the proof of Theorem [T] 

Appendix 

Proof of Theorem |6] 

Follows from the proof of Theorem |2] 

Appendix 

Proof of Theorem |7] 

Follows from the proof of Theorem [T] 

Appendix 

Proof of Theorem |8] 

Follows from the proof of Theorem |2] 



